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Abstract 

We introduce a general learning principle designed to capture the dynamics of an input into a recurrent 
neural network. By minimizing the relative entropy between a given input dynamical system and the 
spontaneous activity of the neural network, we derive a new unsupervised learning rule. This learning rule 
can be interpreted in terms of biologically relevant synaptic plasticity mechanisms, such as spike-timing 
dependent plasticity and synaptic scaling. 

Author Summary 

Learning in the brain is generally thought to be mediated by local changes in the neuronal network 
connectivity occurring at the synapses. The connections between neurons strengthen or weaken according 
to learning rules exclusively based on the activity of the neurons they are connected with. In this paper, 
we introduce a learning rule which gives an explicit way for the connections to tune themselves so that the 
global network learns to copy and reproduce the dynamics of its stimuli. Indeed, it had been advocated 
that the spontaneous activity of a biological neural networks may replay the stimuli the network was 
previously exposed to. Based on this hypothesis, we apply several mathematical tools on a simplified 
network model to identify an optimal learning rule. Although the approach is mainly theoretical, we 
show how the proposed learning rule can be interpreted in terms of biologically plausible mechanisms. 
This suggests a concrete method for the brain to build itself a model of the world. 

Introduction 

For several decades, understanding the synaptic mechanisms underlying learning processes in the brain 
has been an active field of research, starting from the pioneering work of Hebb [1] to the latest discoveries 
on spike-timing dependent plasticity (STDP) initiated by [2, 3]. Learning is mainly thought to rely on the 
modifications of synapses strength caused by co-activations of connected neurons, reflecting in particular 
correlations and causalities in the inputs structure. Beyond these modifications rules, learning is also 
entwined with key homeostatic plasticity processes regulating the overall level of neuronal activity [4, 5]. 

In terms of modeling, two main classes of learning rules can be distinguished: (i) those based on 
biological observations with an aim to clarify their functional purposes, e.g. the BCM learning rule [6], or 
the STDP with a recent interpretation proposed in [7] and (ii) those designed from theoretical principles 
whose link with biology is determined a posteriori [8, 9, 10]. In the second class of learning rules, the 
theoretical principle often corresponds to optimize a certain quantity [11]. The choice of this quantity is 
crucial and leads to qualitatively different results. 

Recent experimental findings [12, 13] have shown that spontaneous activity patterns in the brain 
were very similar to the evoked activity. It suggests that the spontaneous activity collects dynamical 
statistics about its environment and may even replay the stimuli it was exposed to. Although biological 
experiments usually compare spontaneous and evoked activity, it also makes sense from a theoretical 
perspective to compare the spontaneous dynamics of the network directly to the dynamics of its inputs. 
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Therefore, we propose to derive a learning rule which minimizes a specific distance between the 
autonomous activity of a rate-based neural network and another dynamical system which has produced 
the inputs to the network during the learning phase. The distance we choose between the two dynamical 
systems is based on an analogy to the relative entropy or Kullback-Leibler divergence. In the theory of 
stochastic processes, the relative entropy between two diffusion processes appears as a natural way to 
estimate a discrepancy between their laws in the path space. This approach is closely related to the free 
energy principle [14] since it corresponds to minimizing the surprise of the network to its sensory inputs. 

Minimizing this distance may lead to very accurate approximations of the inputs, because it has been 
proved that any dynamical system can be arbitrarily finely approximated by such a network, provided 
there is a sufficient number of neurons [15]. Besides, the minimization of the Kullback-Leibler diver- 
gence between two stationary Gibb's measures, has proven very fruitful in the context of Boltzmann 
machines [16, 17] leading to state of the art methods in data mining. In contrast with Boltzmann ma- 
chines, our choice is tailored to capture dynamical properties of the inputs, rather than static statistical 
features. 

In this article, we intend to derive a biologically relevant learning rule from this theoretical principle. 
First, we express the relative entropy between the input and the network activity as a function of the 
connectivity matrix. Computing the gradient of this quantity with respect to the connectivity leads to a 
gradient descent algorithm to update the connectivity. We then introduce a learning rule approximating 
the gradient descent in the slow-fast asymptotic regime. We also illustrate numerically the properties of 
this learning rule on three examples. Finally, we discuss the biological relevance of the learning rule and 
future extensions. 



Results 

We consider a neural network made of n neurons which is exposed to a time dependent input u(t) e R n 
of the same dimension. As our approach is focused on learning the dynamics of the input, we assume 
that u is generated by an arbitrary dynamical system: 

u = ?(u) (1) 

with £ a smooth vector field from R n to W 1 . We also assume that the trajectory of the inputs is r-periodic. 
Although the key mathematical assumptions on the input u are stationarity and ergodicity, we restrict 
our study to periodic inputs for simplicity. We comment on possible extensions to a stochastic setting in 
the Discussion. 

Although the following method virtually works with any network equations (see Discussion), we 
focus on a neural network composed of n neurons and governed by 

v = -Zv + W.S(v) + u(t) (2) 

where v e R n is a vector representing neuronal activity, W £ M. nxn is the connectivity matrix, I £ R + 
is a decay constant and S is an entry-wise sigmoid function. The spontaneous activity of the network is 
given by the same equation without the term u(t). This setup is illustrated in Figure 1. 



Learning rule derivation 

Inspired from the definition of relative entropy between stochastic processes (see Material and meth- 
ods) we define 

'ds (3) 



/A.: l[ lu(s) + W.S(u( S )) - £(u(s)) 
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When H u = 0, the vector fields of systems (1) and (2) are equal on the trajectories of the inputs. This 
quantity may be viewed as a distance between the two vector fields defining the dynamics of the inputs 
and of the neuronal network along the trajectories of the inputs. Note that it is similar to classical 
gradient methods [11], except that the norms are applied to the flows of inputs and neural network 
instead of their activity. Thus, it focuses more specifically on the dynamical structure of the inputs. 

In order to capture the dynamics of the inputs into the network, it is natural to look for a learning 
rule minimizing this quantity. To this end, we consider the gradient of this measure with respect to the 
connectivity matrix: 



£(u) • S(u)' + lu • 5(u)' - W.S(u) • 5(u)' (4) 



where {x • y'}ij = Xi(s)yj(s)ds for x, y functions from [0, t[ to E™. Thus, an algorithm implementing 
the gradient descent W = — Vw#u is a good candidate to minimize the relative entropy between inputs 
and spontaneous activity. One can easily show that W — > H u is a convex function, thus excluding 
situations with multiple local minima. Moreover, if S(u) ■ S(u)' is invcrtible, one can compute directly 
the minimizing connectivity as W* = [£(u) • S(u)' + lu ■ S(u)'] . [S{u) ■ S(u)'] ~\ 

In terms of biological relevance, there are three main issues linked with equation (4). First, it is a 
batch learning algorithm which requires an access to the entire history of the inputs. Second, it requires 
a direct access to the inputs u, whereas synaptic plasticity mechanisms shall only rely on the network 
activity v. Third, it requires the ability to compute £(u), the time-derivative of the inputs. 

Thus, we want to design a new learning rule mimicking equation (4) which would avoid these problems. 
First, this new learning rule is online, i.e. it takes input on the fly, and relies on a slow/fast mechanism 
to temporally average the variables. Second, it uses an estimate of the inputs, noted v, which is based 
on the activity variable only. Third, it approximates the computation of £(u) with a function S inspired 
from STDP convolution. This learning rule can be written 

-Wy = 5[v u S(y,)] + lviS(vj) - yfikS{v k )S{9j) (5) 
6 k 

where e <C 1 is the learning rate, 5[x, y] = ^(x(y*g 7 ) — (x*g 7 )y) and v = Iv — W.S(v) *gi. The function 
g c is defined as g c : t h- > ce~ ct H(t) where c is a positive number. The constant 7 € E + is a time constant 
corresponding to the width of the STDP window used for learning. A schematic representation of (5) is 
displayed in Figure 2. We give a full justification of the link between this rule and (4) in Materials and 
methods. The online equation (5) is a good approximation of the gradient descent (4) if the following 
assumption is verified 

u ~ u * gi * g 1 (6) 

i.e. the inputs are slow compared to the time scales of the network. If this assumption is not verified 
then the network only learns to replay the slow variations of the inputs. 

A striking feature of this new learning rule is that the three terms appearing in (5) have a biological 
interpretation in terms of STDP, Hcbbian learning and synaptic scaling, as explained in details the 
Discussion. 



Applications 

We now present three examples which illustrate the above learning rule. 
Proof of concept 



In order to illustrate the idea that learning rule (4) enables the network to learn dynamical features of 
the input, we have constructed the following experiment. We present to the network an input movie 
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displaying sequentially the writing of the letter A (figure 3. a - top row). To each pixel we assign one 
neuron, so that the input and the network share the same dimension. This input movie is repeated 
periodically until the connectivity matrix of the network, evolving under rule (4), stabilizes (figure 3.b). 
Then the input is turned off and we set the initial state of the network to a priming image showing the 
bottom left part of letter A. The network evolving without input strikingly reproduces the dynamical 
writing of letter A as displayed in figure 3. a - bottom row. A preliminary investigation of the eigenvalues 
and eigenvectors of the final connectivity matrix reveals that the number of relevant modes (non-zero 
eigenvalue) is slightly below the number of time frames used in the input movie, indicating an efficient 
information storage into the connectivity. Thus, with this example we have illustrated the ability of the 
learning rule we have derived from a theoretical principle to capture a dynamic input into a connectivity 
matrix. 

From activity to connectivity 

We now investigate the question of retrieving the connectivity of a neural network based on the observation 
of the time series of its activity. This is an inverse problem which is one of the most challenging topics 
in computational neuroscience [18, 19, 20] since it may give access to large scale effective connectivities 
simply from the observation of a neuronal activity. Here we address it in an elementary framework. 
The network generating the activity patterns is referred as input network and evolves according to 
u = — Z u + Wq.S(u). For this example, the network is made of n = 3 neurons and its connectivity W 
is shown in figure 4. a). These parameters were chosen so that the activity is periodic as shown by the 
dashed curves in figure 4.c). Then, we simulate both equations (2) and (5) with a decay constant l net 
and observe that its connectivity W ne t converges to Wq. 

As opposed to the batch learning rule (4), it is necessary for assumption (6) to be satisfied in order 
to approximate accurately the input's activity with the online learning rule (5). Given that the time 
constant of the inputs is governed by lo, the previous approximation holds only if j,l ne t S> lo- If this 
assumption is broken, e.g. l net — lo, then the final connectivity matrix si different from Wo, see the red 
dot-dashed curve in figure 4.b). Indeed, in this case, the network only learns to replay the slow variation 
of the inputs. 

A method to recover the precise time course of the inputs consists in artificially changing the time 
constants at different steps of an algorithm described in the following. First, simulate the network 
equation (2) with a constant l net ~> lo- Yet, the time constant in the learning equation (5) is to be kept 
at lo, thus introducing a hybrid model. In this framework, the connectivity converges exactly to Wo as 
shown in the blue dashed curve in figure 4.b). After learning, simulate the network equation (2) with 
the learned connectivity and with a time constant switched back to lo- This gives an activity as shown 
in the plain curves in figure 4.c). 

Learning the dynamics 

The last example extends the approach by devoting an arbitrarily large number of neurons to the process- 
ing in contrast to the two previous examples. Indeed, the computational power of neural networks of the 
type (2) lies in the possibility of having a huge amount of neurons. However, the learning rule (5) implic- 
itly assumes that the network and its inputs have the same dimensionality Indeed, if a neuron receives 
no input then a close inspection of (15) shows that, at equilibrium, it does not receive connections from 
other neurons. Therefore, learning low dimensional, yet temporally rich, inputs leads to poor results. To 
overcome this technical issue, low dimensional inputs need to be projected in higher dimensions before 
they can be accurately learned by the recurrent neural network introduced above. An example of such a 
projection, corresponds to considering that neurons code for the position of the input in its phase space. 
For instance, a neuron might receive excitation only when the input is in a small ball centered around 
the origin. This projection somewhat corresponds to the discretization of the transport equation equa- 
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tion derived from (1). The left picture of figure 5.b illustrates this projection for the following example. 
Assume that the input trajectory is given by a point [x{t), y(t)) = ( — 3 cos(t), sin(t) + cos(2t)) as shown 
by the blue curves in the top picture of figure 5. a. This particular input is chosen because its trajectory 
in the phase space does not intersect itself and thus it is the solution of a dynamical system (which we do 
not need to explicit). Besides, if we try to approximate this two dimensional input with a network of two 
neurons, then the spontaneous activity post-learning is a very bad approximation of the signal, as shown 
by the red curves in the middle picture of figure 5. a. Indeed, the two-neurons networks of type eqrcfeq: 
voltage based do not span the dynamics provided by the inputs (which has different mode of oscillations) . 
However, we show below that increasing the number of neurons leads to a much better approximation as 
shown in the bottom picture of figure 5. a. 

In the high dimensional space it is possible to compare the dynamics of the projected inputs p(u) 
and the dynamics v as d(u,v) = ^ J? \\p(u(s)) v(s)\\ x ds = ^ ELo \\p«~¥)) v(^)||i, 
where n e f / is the number of neurons which receive a non-null input and T = 50 is the number of time 
steps used for discretization. In the right picture of figure 5.b, we show that the accuracy decreases to 
zero with the number of neurons in the network. When the error is null we see on the red curve that the 
number of non-quiet neurons is always equal to T, which means that every point of the inputs has been 
"boxed" by separate neurons. 

The equilibrium connectivities for different number of neurons are represented in figure 5.c. We observe 
that when the number of neurons is large enough for the representation of the input to be optimal (at 
machine precision), i.e. the right picture corresponding to n — 3364 neurons, the neurons are strongly 
self inhibitory and send their excitation to one neuron. Actually, they pass their excitation to the next 
neuron and become immediately quiet due to self inhibition. When there are less neurons in the network, 
i.e. the left picture corresponding to n = 100 neurons, the behavior of the network is similar except 
that the self inhibition of some neurons is not sufficient to suppress their excitation in one time step. 
Therefore, they will be excited during a longer time. This corresponds to the fact that two different time 
steps of the inputs lead to exciting the same neuron. Therefore, the accuracy is poorer, see right picture 
of figure 5.b. 

The projection we introduced is a classical precomputation trick used in many algorithms. We suggest 
that it may correspond to the processing performed by vertical connections connecting the senses to the 
cortex, e.g. retino-cortical connections. Indeed, in the primary visual cortex it is well-known that some 
neurons code for local oriented edges in the visual field. The receptive fields of the neurons cover all the 
space position plus orientation. 

Discussion 

Biological intrepretation 

The learning rule (5) introduced in this article can be interpreted from a biological point of view. We 
now discuss the different terms in (5) and relate them to the biologically oriented litterature. 

The variable v can be seen as a spatio-temporal differential variable. Although unsupervised learning 
rules are often algebraic combinations of element-wise functions applied to the activity of the network [21], 
it is not precisely the case here. Indeed, learning is based on the variable v which corresponds to the 
subtraction of the temporally integrated synaptic drive W.S(v) * g/from the activity of the neurons Iv. 
For each neuron, this variable takes into account the past of all the neurons which are then spatially 
averaged through the connectivity to be subtracted from the current activity. This gives a differential 
flavor to this variable which is reminiscent of former learning rules [6, 22] for the temporal aspect and [23] 
for the spatial aspect. Note that this variable is not strictly speaking local (i.e. the connection Wjj needs 
the values of to be updated), yet it is biologically plausible since the term (W.5(v)) is accessible for 
neuron i on its dendritic tree, which is a form of locality in the broader sense. 
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The first term 5[vi, S(vj)] in (5) can be related to antisymmetric STDP learning [24]. This term 
is antisymmetric and is responsible for retrieving the drift £ in the mathematical computations above. 
Thus it captures the causality structure of the inputs which is a task generally attributed to STDP [25] . 
Beyond the simple similarity of functional role, we believe a simplification of this term may shed light 
on the deep link it has with STDP. The main difference between our setup and STDP is that the former 
is based on a rate-based dynamics (2), whereas the latter is based on a spiking dynamics. In a pure 
spike framework, i.e the activity is a sum of Diracs, the STDP can be seen as this simple learning rule 
Wjj oc 5[vi, Vj] oc v,(vj * <7 7 ) — (vj * g 7 )vj- Indeed, the term v,(vj * g 7 ) is non-null only when the post- 
synaptic neuron i is firing and then, via the factor Vj * g 1 , it counts the number of preceding pre-synaptic 
spikes that might have caused i's excitation and weight them by the decreasing exponential g 1 . Thus 
this term exactly account for the positive part of the STDP curve. The negative term — (vj * fl , 7 )vj takes 
the opposite perspective and accounts for the negative part of the STDP curve. A loose extension of this 
rule to the case where the activity is smoothly evolving leads to identifying the function 5 to the STDP 
mechanism for rate-based networks [26, 27]. 

The second term lviS(vj) in (5) is a form of Hebbian learning. This term is similar to the original 
Hebbian learning rule [1, 28, 29]. It corresponds to the co-activation of the pre- and post-synaptic 
cells. Actually, it can also be seen as the symmetric part of the STDP temporal profile complementing 
the purely antisymmetric STDP of the previous term . This is better seen if one slightly changes the 
learning rule (5) by replacing this term by Z(vj * <? 7 ) (S(vj) * g 7 ) which would not change the qualitative 
behavior of the learning rule, e.g. it would still minimize the relative entropy. In this case and based 
on the Material and methods, it is easy to combine it with the S function such that it becomes 
<5[x,2/] = * g~f) — ^-(x * g 7 )y, which corresponds to an asymmetric STDP learning rule [24]. 

The third term — ^ fc W ifc S , (v fe )S'(v : ,) in (5) accounts for what is usually presented as homeostatic 
plasticity mechanisms. Although the two previous terms (STDP and Hebbian learning) seem to be 
powerful mechanisms to shape the response of the network, there is a need of a regulatory process to 
prevent from uncontrolled growth of the network connectivity [4, 5, 30]. It has been argued that STDP 
could be self regulatory [31, 32], but it is not the case in our framework and an explicit balancing 
mechanism is necessary to avoid the divergence of the system. This last term is the only one with 
a negative sign and is multiplicative with respect to the connectivity. Thus, according to [4], it is a 
reasonable candidate for homeostasis. It has been argued [33, 34] that homeostatic plasticity might keep 
the relative synaptic weights by dividing the connectivity with a common scaling factor, theoretically 
preventing from a possible information loss. In contrast to these ad hoc re-normalizations often introduced 
in other learning rules [23, 35], our relative entropy minimizing learning rule thus introduces naturally 
an original form of homeostatic plasticity. Although we have separated the description of the three terms 
in (5), our approach suggests that homeostasis may be seen, not necessarily as a scaling term, but as a 
constitutive part of a learning principle, deeply entangled with the other learning mechanisms. 

Extension and perspectives 

Although our approach eventually aims at addressing stochastic inputs, there are several conceptual 
and technical issues to extend our approach to noisy versions of (1) and (2). In the present article, we 
have introduced the quantity H u to measure the distance between the dynamics of the inputs and the 
spontaneous activity of the network after learning. This quantity is based on an analogy with the notion 
of relative entropy between the laws of two solutions of different stochastic differential equations sharing 
the same diffusion coefficient. Yet, the definition does not hold for different diffusion coefficients which 
is, a priori, our case. Thus, one may think that the network should learn both the vector field through 
W and the diffusion matrix. Indeed, the network should learn also the covariant statistics of the input 
fluctuations to be able to reproduce similar patterns after the learning phase. An interesting perspective 
in this direction might be to implement a form of fluctuation-dissipation relationship to extract the 
relevant input statistics with an online learning rule. Although promising from an algorithmic viewpoint, 
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biological relevance of such a form of dual learning seems difficult to establish. Another technical point 
is that our construction of W through (5) relies on the use of a STDP-inspired learning to estimate the 
time derivative £(u) of the inputs. However, if the inputs are perturbed by noise, then this estimation 
becomes less accurate. 

Moreover, it appears that our general approach can be applied to other forms of network equations 
than (2) such as the Wilson-Cowan or Kuramoto models, leading to different learning rules. In this 
perspective, learning can be seen as a projection of a given arbitrary dynamical system to a versatile 
neuronal network model, and the learning rule will depend on the chosen model. However, we shall 
remark that any network equation with an additive structure - intrinsic dynamics + communication 
with other neurons - as in (2) will lead to a similar structure for the learning rule, with three terms 
that may share similar biological interpretations as developed above. A special case is the linear network 
v = — Iv + W.v + u(i) for which various statistical methods to estimate the connectivity matrix have 
been applied e.g. in climate modeling [36], gene regulatory networks [37] and spontaneous neuronal 
activity [19] . The method developed in this article may be used to extend the inverse modeling approach 
previously developed in the linear case to models with non-linear interactions. 

Conclusion 

In this article, we have identified a theoretical link between various highly debated concepts in neuro- 
science : Hebbian learning, spike-timing dependent plasticity and synaptic scaling. We have proved that 
an appropriate combination of these local mechanisms leads the network to learn to replay and predict 
its inputs. Not only this article focuses on the biological plausibility of the new learning rule, but it also 
provides a constructive way to build a network which reproduces the dynamics of any ergodic system. 



Materials and Methods 
Relative entropy 

The quantity minimized through learning is defined in (3) and can be interpreted as the measure of the 
distance between the flows of the inputs and the neural network along the trajectories of the inputs. 
Although it is the starting point of our analysis, it is related and inspired by the relative entropy or 
the Kullback-Lcibler divergence between two stochastic processes. These two stochastic processes are 
assumed to be the solution of a stochastic differential equation with the same diffusion term. This point 
explains why we could not apply directly the relative entropy minimization to our problem, since there 
is no reason why inputs and neural network should share the same diffusion term. Therefore, we restrict 
our study to deterministic systems and borrow the analogous of relative entropy to the field of stochastic 
processes. 

We now introduce the computation of the relative entropy between two real diffusion processes x? 
and x 9 sharing the same diffusion coefficient. The extension to multidimensional stochastic processes is 
straightforward. Let {f2, P, {J r t}o<t<T} a probability space, equipped with the natural filtration of 
the standard Brownian motion. Consider a diffusion process (x t ) in R under P, solution of the following 
stochastic differential equation (SDE): 

dx t = f(x t )dt + adB t (7) 

where B t is a P-brownian motion. By Girsanov theorem, if Q is another probability absolutely continuous 
with respect to P and such that: 

dQ ( [ T g{x s )-f{x s ) 1T3 1 f T 



j(x 3 ) - f(x s ) 



ds 



(8) 
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then, under the technical condition that 
under Q, solution of : 

dxt — g(x t )dt + adB t 



ds < oo P-a.s, one obtains a diffusion process 



(9) 



where B t is a Q-brownian motion. In other words, changing the drift in a diffusion process with a constant 
diffusion coefficient is equivalent to a change of the underlying probability. Notice that if / = g, then 



|2 = 1 and P = Q. 



We define then the relative entropy entropy between the two diffusions 

dx{ = f(x{)dt + adB t 



and 



dxf = g(x 9 )dt + adB t 



as the relative entropy between Q and P: 

H(x 9 \x f ) = H{Q\P) :=E Q 
An elementary computation shows that: 

T \g(xl)~f(xiy 2 



H(x 9 \x f ) = -E 



/ 

J o 



ds 



(10) 
(11) 

(12) 
(13) 



Similarly we can define H{x^\x 9 ) by exchanging / and g, which gives a different quantity. Therefore the 
relative entropy between two stochastic processes is not a distance. Yet it gathers interesting information 
about the similarity between the two processes. A limitation of this definition is that it can only be defined 
in the case where both diffusion processes x? and x 9 share the same diffusion matrix a. Otherwise the 
two associated measures P and Q are not absolutely continuous with respect to each other. However, 
this definition can interestingly be extended to the case of noise-free dynamical systems, which is the 
starting point of our definition of H u in equation (3). 



Justification of the online learning rule 

The online learning rule (5) is expressed by means of the activity of the neural network v governed by 
(2). However, to be comparable to the gradient of the relative entropy (4), we first need to make explicit 
the dependency on the inputs u. Therefore, we show that the network dynamics induces a simple relation 
between v and the inputs u: a simple computation in the Fourier domain shows that the convolution 
between the temporal operator + 11^ and gi results in 11^. Applying this result to the neural dynamics 
leads to reformulating equation (2) as Zv — W • S(v) * gi = u * gi. We recognize the definition of the 
variable v (which was originally defined according to this relation) such that the network's dynamics (2) 
is equivalent to 

v = u * gi (14) 

To prove that (5) implements the gradient descent of the relative entropy, we need to use a time-scale 
separation assumption, enabling the input to reveal its dynamical structure through ergodicity. Indeed, 
as learning is very slow compared to the activity v we can assume that the learning rate e is very small 
compared to 1. In this case, we can apply classical results of periodic averaging [38] to show that the 
evolution of W is well-approximated by 



W = [v * A 7 + Iw - W.S(v)] • S(v)' 



(15) 
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where A 7 : 1 1->- ^ (<? 7 (— i) — 9-y(t)) ■ In the equation above, v is the solution of equation (2) such that we 
can use relation (14). Besides, by going into the Fourier domain, we prove (see below) that 



[x * A 7 ] ■ y' = (i * ,g 7 ) • (y * g 7 )' 
Therefore, with equation (1), the averaged equation above becomes 



W = [£(u) * 9l ] ■ [S(u) *g 7 ]' + lu- S(u)' - W • 5(u) • S(u)' 



where u = u*gi. If u is slow enough, i.e. u*gi*g 1 ~ u, then this equation is precisely the gradient descent 
of H u . If u is too fast, then the network will only learn and replay the slow variations of the inputs. 
Some fast-varying information is lost in the averaging process, mainly because (2) acts as a relaxation 
equation which filters the activity. Note that this loss of information is not necessarily a problem for 
the brain, since it may be extracting and treating information at different time scales [39] . Actually, the 
choice of the parameter I specifies the time scale under which the inputs are observed. 

Stability, convergence to a fixed point of the learning rule (5) follows immediately by applying 
Krasovskii-Lasalle invariancc principle [40]. 

STDP as a differential operator 

We justify here the following equality [x * A 7 ] • y' = (x * g 7 ) • (y * g 7 )' . This is the key mathematical 
mechanism that makes STDP a good approximation of the temporal derivative of the inputs. 

We proceed in two steps, both of which consist in going in the Fourier domain, where convolutions 
are turned into multiplications and reciprocally. We use the the unitary, ordinary frequency convention 
for the Fourier transform. Observe that the Fourier transform of g 7 is <? 7 (£) = 7 rpi^f| ■ And we define 

g'j-t^ g-y{-t) such that A 7 = ^(g' y - g 7 ) and p 7 (£) = ff 7 (-£). 

1. Let us show that x * A 7 = x * 9l . 

The Fourier transform of the convolution x * A 7 is the product x A 7 . Besides, 



2 i 



A 7 = ff 7 -3 7 (0 = 



7 7 



= 2i?r£ 



27 



7 



7 — 2«7r£ 7 + 2i7r£ 



r 



2 + 4tt 2 £ 2 




Because ^jr(£) = x 2z7r£, taking the inverse Fourier transform of x 2in^ 9 ''^ 2 9 '" (g) gives the result. 



2. Let us show that [x * 9l \ 9l \ • y' = (x * g 7 ) • (y * g 7 )' 



We proceed again in two steps: 




(b) Let us show that (x * g' 7 ) • y' = x • (y * g 7 )' 
Compute 
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Using the result (a) and applying result (b) to x * g 1 and y leads to the result [x * 9 ~' \ 9l \ ■ y' = 
(x*g 7 ) • (y * g 7 )'. 

Using the first result and applying the second to x and y leads to the result [x* A 7 ] -y' = (x*g 7 ) • (y*<7 7 )'. 
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Figure Legends 




evolving connectivity Wy 

other contributions (synaptic scaling) 



Figure 1. Schematic picture of the neuronal network. Each neuron receives an external input 
and is connected to other neurons in the network. If the connection strength W« (pink) from neuron j 
to neuron i evolves according to learning rule (5) , then neurons i and j contributes to the STDP 
convolution and Hebbian learning parts, while other neurons ki contributes to the synaptic scaling 
through the synaptic drive received by neuron i, thus locally accessible by neuron i. 
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STDP convolution 




S(vj) 













Synaptic scaling 

EWikS(v k ) S(Vj) 




Figure 2. Schematic representation of the learning rule (5). The connection strength Wy 
between neuron j and neuron i is dynamically modified according to the right-hand side of (5). The 
first term can be seen as a convolution with a SDTP-likc filter (green box). The second term appears in 
the form of a product between the activity of neuron i and the post-synaptic activity of neuron j, 
therefore called Hebbian rule (blue box). Eventually, the last term involves a product between the 
synaptic drive received by neuron i and the post-synaptic activity of neuron j. Modifying Wy with a 
negative sign, this term may account for synaptic scaling mechanisms. See the Discussion for a 
detailed biological interpretation of each term. 
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Figure 3. Learning how to write the letter A. a) Time evolution of the input movie (top row) 
and of the network activity after learning (bottom row). Each pixel corresponds to a neuron, b) 
Evolution of the connectivity matrix at different times during the learning phase. Parameters: Number 
of neurons n — 400; I = 1; S(x) — tanh(x). 
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time 100 

Figure 4. Retrieving the connectivity a) Connectivity matrix Wo of the input network, b) 
Evolution of the difference between current connectivity and input connectivity through learning. The 
red dot-dashed black curve corresponds to the online learning rule (5) in the homogeneous case, i.e. 
I = Inet = lo m (5) and (2). The blue dashed curve corresponds to the online learning rule (5) in the 
hybrid framework, i.e. I = Iq in (5) and I = l ne t S> Iq in (2). For this simulation we chose l ne t = 50. The 
black curve corresponds to the batch relative entropy minimization (4). c) The dashed curves 
correspond to the activity of the inputs. It is a three dimensional input and the three different colors 
correspond to the different dimensions. The plain curves correspond to the simulation of network (2) 
post learning, in the hybrid framework, and without inputs. The parameters for these simulations are 
l = l,e = 0.01 and 7 = 100. 
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Figure 5. Learning the low dimensional inputs with high dimensional networks a) (top row) 
The inputs are displayed in blue, (middle row) The naive application of the learning rule (4) to a two 
dimensional network leads to the approximation in red. (bottom row) The reconstruction of the inputs 
for a network copying the dynamics in the phase plane. This corresponds to plotting the different 
components of the observed trajectory in the phase plane which is created autonomously by the 
network post-learning. We used n — 3364 neurons in this figure, b) (left) Discretization of the phase 
space of the inputs. The inputs are plotted in blue in their phase space. Only the neurons 
corresponding to the red boxes are active during learning, (right) In blue, the accuracy of the 
approximation post-learning for different number of neurons in the grid above. In red, the number of 
neurons really involved in learning n e ff corresponds to the number of red boxes in the left picture of 
figure b. c) Equilibrium connectivity for different number of neurons in the network: n = 10 2 = 100 
neurons for the left picture and n = 58 2 = 3364 for the right picture. The irregularity (non 
convolutionallity) of the connectivity is due to the fact a circle is irregularly projected on the 
rectangular grid. The parameters used in this simulation are I = 10. for figure a and I = 0.1 for figures b 
and c, S(x) — tanh(x) and the input is temporally discretized on T = 50 points. 



